Beyond the mask: communicating emotions in the era of COVID-19

Three weeks into mentoring a new colleague, Dolly Bindon — a hospital nurse in Colorado — finally ate lunch with her trainee for the first time. They sat down at an outdoor table and pulled down their masks.

“I saw the bottom half of her face for the first time,” says Bindon. “We were together three days a week, 12 hours a day, for three, four weeks. And I was like, ‘Huh, who knew? That’s what your face looks like?’”

It’s been almost a year since the World Health Organization declared COVID-19 a global pandemic, and with the Biden administration in place, masks aren’t going away anytime soon. Every U.S. state but South Dakota requires masks in at least some public settings or local districts. Even basic cloth face coverings likely reduce transmission of the virus by 30% or more, and without widespread vaccination, masks remain one of our best lines of defense. But masks aren’t without their downsides. When worn correctly, masks make it more difficult for us to read others’ expressions.

With the mouth, cheeks and chin obscured, it’s harder to recognize and interpret other people’s faces and emotions. Without emotional recognition, conversations are uninflected, “just like reading words on a page with no punctuation,” says David Matsumoto, a psychologist who studies nonverbal expression at San Francisco State University.

Fortunately, there’s a rich body of psychological research that can help us navigate this strange moment — a time when we often can only guess what someone is thinking behind the mask. Here are some questions and answers about the science of facial expression and recognition.

How do people communicate emotion?

With words, people can directly say what they feel. But we also rely heavily on non-verbal cues: tone of voice, gesture, touch, body language and facial expressions. “The channel that communicates the most specific emotional states is the face,” Matsumoto says.

In the 1970s, psychologist Paul Ekman classified a set of basic emotions. To be considered basic, an emotion has to be universal across cultures, expressed through certain shared facial expressions, and spontaneous — meaning that it happens organically in response to an appropriate trigger. “Like, for example, you see a bear, you’re scared,” explains Lisa Aziz-Zadeh, a cognitive neuroscientist at University of Southern California. Today, most experts in psychology and neuroscience agree that there are at least six of these basic emotions: happiness, sadness, fear, disgust, anger and surprise.

Expressing a basic emotion gets the full face involved. “It’s not just in the mouth… there’s an associated pattern of muscles for the eyes, for the cheeks, for the eyebrows,” says Aziz-Zadeh. Surprise and fear trigger the eyes to open wider, while disgust is shown in a wrinkle at the bridge of the nose. Happiness is expressed in the corners of the eyes and upper cheeks. Some of these basic emotions share facial trademarks, like the close-knit brows of disgust and anger, but there are enough unique characteristics of each — especially when the whole face can be seen and assessed — that most people can tell these expressions apart.

What gets lost with masks?

Not all emotions are shown in every part of the face, though. Contempt, sometimes also considered a basic emotion because of its universality, is often only demonstrated through a small twist of the mouth, Matsumoto says. More subtle expressions, combinations of emotions, or the intensity of an emotion can all become much harder to interpret when the lower face is covered.

“If you’re covering the face, you are impoverishing the whole visual experience,” explains Roxane Itier, a cognitive neuroscientist at University of Waterloo in Ontario. The time it takes to read other people’s facial expressions likely increases when masks are worn, says Itier, while the accuracy of those interpretations goes down. “Humans didn’t evolve to read half the face,” agrees Matsumoto.

Further, people on the autism spectrum may struggle more than neurotypical people with masked communication. Some people with autism focus their attention mostly on the mouth, and “explicitly avoid looking at the eyes,” says Aziz-Zadeh. To better communicate with people who rely on the mouth for emotional cues, including people who are deaf or hard-of-hearing, Aziz-Zadeh recommends masks with clear inserts. Clear masks are generally more expensive, however, and less readily available than standard masks.

What can we do to compensate?

Masks save lives — by one analysis, 95% mask compliance could prevent hundreds of deaths per day in the U.S. — so sacrificing proper mask wearing just to see a stranger’s smile is not a good trade-off. 

But Matsumoto says all is not lost. “I think that there’s good things to be said about communicating well, even with masks,” he says. With all the other modes of emotional communication in play, missing out on the mouth doesn’t have to stop humans from understanding each other.

“People are very ingenious in finding ways to compensate and adapt to the environment, because communication is so, so vital,” he explains. People can use touch or gestures to emphasize emotion, says Matsumoto, and even directly explain how they’re feeling. Telling someone that you’re smiling, surprised or upset is an efficient way to make sure they understand you, and we do it all the time in phone conversations.

Further, remember that even in a non-pandemic reality, facial expressions are rarely the sole means of nonverbal communication. “We also use context enormously,” says Itier. For example, that scary bear might elicit a similar face to walking in on your own surprise party, but observers would interpret those expressions differently. Picking up on conversational context clues is often unconscious, but you can purposely increase your observational efforts as well by actively listening.

If you’re still concerned with understanding people’s emotions and sharing your own, you could purchase a clear mask, focus more on tone of voice, or you could just look to the eyes, which are key in expressing all of those six basic emotions. After all, in professions like nursing, masks have long been commonplace — and health care workers still manage to communicate effectively.

Even so, Dolly Bindon says things do feel different at her Colorado hospital now. Some of the lighthearted exchanges she’s used to don’t translate as well when both patients and health care workers are masked.

“We have to ask patients every day: ‘Where are you? What’s your name? What day is it?’ And sometimes they’ll be making a dad joke, which has always happened in the history of every patient in the hospital ever. And they pretend that they don’t know what day it is… But like, I don’t know, if you’re wearing a mask — [are you] joking? Are you serious? I don’t know.”

Can you see beyond the mask? Take your best guess at what face each person is making, and then move the slider to see what you missed.

1)

Answer Key:

1) Anger. Visible above the mask: low, furrowed brow. Hidden under the mask: tightened lips and flared nostrils.

2) Disgust. Visible above the mask: wrinkled upper nose, knit brow and narrowed eyes. Hidden under the mask: lower nose wrinkle and grimace.

3) Fear. Visible above the mask: widened eyes with visible whites and raised brows. Hidden under the mask: flared nostrils and downturned mouth.

4) Happiness. Visible above the mask: upturned corners of the eyes and raised cheeks. Hidden under the mask: wide smile.

5) Sadness. Visible above the mask: slightly knit brows, downturned corners of the eyes. Hidden under the mask: frown or pout.

6) Surprise. Visible above the mask: slightly raised eyebrows and widened eyes. Hidden under the mask:open mouth. Without the lower face, it can be hard to tell surprise from fear, so context is important.

7) Contempt. Visible above the mask: eyes and brow look neutral, so contempt is difficult to spot above the mask. Hidden under the mask though, the sideways mouth position signals contempt.

[Images by Lauren Leffer ©2021.]